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In re patent application of , 

Joshua Goodman 
10/766,348 
January 28, 2004 

EXPONENTIAL PRIORS FOR MAXIMUM ENTROPY MODELS 
2857 

Paul L. Kim 

Supplemental Information Disclosure Statement 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Sir: 

1 . Pursuant to 37 C.F.R. £1 .97 and 41 .98, and in compliance with 37 C.F.R. £1 .56, the Office's attention is directed to 
the patents, publications and other information listed on the attached PTO-1449. A copy of each listed document is enclosed 
except for: (a) pending applications; (b) those previously cited or submitted to the Office in the following application(s) upon 
which this application relies for an earlier filing date under 35 U.S.C. §120; and/or (c) U.S. Patent(s) and published 
applications(s). 

Serial No.: 

Filing Date: 

Regarding any document, publication or other information for which a date is not given on the attached PTO-1449, 
Applicant(s) believe(s) the same may qualify as "prior" art to this application and should be treated accordingly, although 
Applicant(s) reserve(s) the right to contest the prior art status of any document, publication or information, should issue arise. 

2. Regarding each listed document that is not in the English language, an English-language translation accompanies 
this Statement as indicated on the attached PTO-1449 or a concise explanation of the relevance of the document is set forth in 
the following document(s): 

(a) O Copy of each English language version of a search report indicating the degree of relevance found by the 

foreign office of each document being submitted from the search report. 

(b) □ Attachment entitled "Concise Explanation of Relevance of Non-English Language Documents". 

3. Pursuant to 37 C.F.R. 41 .97(b) this Statement is being filed (one must be checked): 

(a) [H Within 3 months of the filing date or date of entry into the National Stage. 

(b) □ Before the mailing date of a first Office Action on the merits. If this Statement is not filed before the 

mailing date of a first Office Action on the merits, the required certification is given below or, in the 
absence thereof, the Office is authorized to charge the required fee set forth in 37 C.F.R. £1.1 7(p) to 
Deposit Account No. 50-1063 for consideration of this Statement. 

(c) □ Before the mailing date of an Office Action after the filing of a request for continued examination under 

1.114. 
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(d) ^ After the period set forth in 37 C.F.R. £1 .97(b) but before the mailing date of either a final action or a 

notice of allowance. 

(1) D The required certification is given below, or 

(2) S Regarding the fee set forth in 37 C.F.R. £1 . 17(p) for consideration of this Statement : 

(0 CD Enclosed is a check covering the fee; 

(ii) □ Charge the fee set forth in 37 C.F.R. fil.l7(p) to Deposit Account No. 50-1063; or 

(iii) El Enclosed is a credit card payment form covering the fee. 

(e) □ After the mailing date of either a final action or a notice of allowance, but before payment of the issue fee. 

Petition hereby is made for consideration of this Statement and the required certification is indicated below. 
Regarding the fee set forth in 37 C.F.R. §1.17(i)(l) for consideration of this Statement: 

(1) □ Enclosed is a check covering the fee; 

(2) □ Charge the fee set forth in 37 C.F.R. £1.17(i)(l) to Deposit Account No. 50-1063; or 

(3) □ Enclosed is a credit card payment form covering the fee. 

4 . Certification (if applicable) 

(a) D The undersigned hereby certifies that each item of information contained in this Statement was cited in a 

communication from a foreign patent office in a counterpart foreign application not more than 3 months 
prior to the filing of this Statement. 

(b) □ The undersigned hereby certifies that no item of information contained in this Statement was cited in a 

communication from a foreign patent office in a counterpart foreign application or, to the undersigned's 
knowledge after making reasonable inquiry, was known to any individual designated in 37 C.F.R. §1. 56(c) 
more than 3 months prior to the filing of this Statement. 

5. The Commissioner is hereby authorized to charge any additional fees or credit any overpayment to Deposit Account 
No. 50-1063. 

Respectfully Submitted, 



By. 

Himanshu S. Amin 
Reg. No. 40,894 

Amin & Turocy, LLP 
24 th Floor, National City Center 
1900 East 9 th Street 
Cleveland, Ohio 44114 
(216) 696-8730 

CERTIFICATE OF MAILING 
I hereby certify that this correspondence (along with any paper referenced as being attached or enclosed) is 
being deposited on the below date with the United States Postal Service with sufficient postage as first class 
mail in an envelope addressed to Commissioner for Patents, P.O. Box 1450. Alexandria, VA 22313-1450. 

Date 
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Cite 
No. 1 



Include name of the author (in CAPITAL LETTERS), title of the article (when appropriate), title of the item (book, 
magazine, journal, serial, symposium, catalog, etc.), date, page(s), volume-issue number(s), publisher, city 

and/or country where published. 



M. BANKO and E. BRILL. Mitigating the Paucity of Data Problem: Exploring the Effect of Training Corpus Size on 
Classifier Performance for NLP. In Proc. of the Conference on Human Language Technology, 2001 . 5 pages. 



A.L. BERGER, et al. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, 22(1): 
39-71, 1996. 



S.F. CHEN and R. ROSENFELD. A Survey of Smoothing Techniques for ME Models. IEEE Transactions on Speech and 
Audio Processing, Vol. 8 No. 1, Jan. 2000. 14 pages. 



S. DELLA PIETRA, et al. Inducing Features of Random Fields. IEEE Transactions on Pattern Analysis and Machine 
Intelligence, 19(4): 380-393, 1997. 



I.J. GOOD. The Population Frequencies of Species and the Estimation of Population Parameters. Biometrika. Vol. 40 No. 
3/4, pp. 237-264, 1953. 



J. GOODMAN. Classes for Fast Maximum Entropy Training. In ICASSP 2001. 4 pages. 



CM. KADIE, et al. CFW: A Collaborative Filtering System using Posteriors over Weights of Evidence. In Proc. of UAI, 
pp. 242-250, 2002. 



R. KNESER and H. NEY. Improved Backing-off for M-gram Language Modeling. In ICASSP, Vol. I, pp. 181-184,1995. 



W. NEWMAN. An Extension to the Maximum Entropy Method. IEEE Transactions on Information Theory, Vol. IT-23, 
No. I, January 1997. 5 pages. 



J. DARROCH and D. RATCLIFF. Generalized Iterative Scaling for Log-linear Models. The Annals of Mathematical 
Statistics, 43: 1470-1480, 1972. 
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•EXAMINER: Initial if reference considered, whether or not citation is in conformance with MPEP 609. Draw line through citation if not in conformance and not 
considered. Include copy of this form with next communication to applicant. 

1 Applicant's unique citation designation number (optional). 2 Applicant is to place a check mark here if English language Translation is attached. 
This collection of information is required by 37 CFR 1.97 and 1.98. The information is required to obtain or retain a benefit by the public which is to file (and by the 
USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 2 hours to complete, 
including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments 
on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent 
and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS 
ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450. 



If you need assistance in completing the form, call 1-800-PTO-9199 and select option 2. 
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S.F. CHEN and J. GOODMAN. An Empirical Study of Smoothing Techniques for Language 
Modeling. Computer Speech and Language, 13: 359-394, October 1999. 
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J. REYNAR and A. RATNAPARKHI. A Maximum Entropy Approach to Identifying Sentence 
Boundaries. InANLP, 1997. 4 pages. 








R. ROSENFELD. Adaptive Statistical Language Modeling: A Maximum Entropy Approach. PhD 
Thesis, Carnegie Mellon University, April 1994. 114 pages. 








S. KHUDANPUR. A Method of Maximum Entropy Estimation with Relaxed Constraints. In 1995 
Johns Hopkins University Language Modeling Workshop, 1995. 18 pages. 








P.M. WILLIAMS. Bayesian Regularization and Pruning using a Laplace Prior. Neural 
Computation, Vol. 7, pp. 117-143, 1995. 
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1 Applicant's unique citation designation number (optional). 2 Applicant is to place a check mark here if English language Translation is attached. 
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to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 2 hours to complete, including 
gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the 
amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and 
Trademark Office, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: 
Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450. 
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